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DYNAMIC ANNOTATIONS FOR ELECTRONIC 

DOCUMENTS 

BACKGROUND 

This invention relates to applying annotations to electronic documents. Electronic 
documents are collections or aggregations of electronic data, such as files or collections of files, 
that typically can be displayed with an appearance that resembles that of a paper document. 
Early electronic document applications - that is, applications in which electronic documents can 
be created, manipulated and viewed, in any combination — such as word processors, publishing 
applications, and the like, allowed a user to do little more than add and change text on an 
electronic document page with simple formatting selections. As electronic document 
applications have evolved, the number of options to alter the appearance of electronic documents 
has grown. Some applications allow users to add annotations, such as stamps, to an electronic 
document. One such electronic document application, Adobe® Acrobat®, available from Adobe 
Systems Incorporated of San Jose, California, allows a user to apply one or more of a number of 
stamps. The stamp can display a word, such as "Confidential", "Draft", "Approved" or any of a 
number of other available selections. 

Adding a stamp to an electronic document can be as simple as selecting a stamping tool 
with a pointing device, moving a cursor controlled by the pointing device to the area of the 
document where the stamp is to be applied, and activating the pointing device. The selected 
stamp is applied by the application. The stamp can be altered by modifying one or more stamp 
properties — for example, selecting a new appearance for the stamp, dragging the stamp to 
another section of the document or deleting the stamp. 

SUMMARY 

The invention provides computer-implemented techniques for applying a dynamic 
annotation to an electronic document. According to the techniques described herein, an instance 
of an annotation can be applied to the electronic document based on existing conditions at the 
time the instance is applied. When two of the same annotation are applied, the appearance of 
each annotations may be the same or different, depending on the state of the conditions upon 
which the annotations are based when the annotations are applied. 
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In general, in one aspect, the invention features methods and apparatus, including 
computer program products, implementing techniques for annotating an electronic document in 
an electronic document application. A request to add an annotation to an electronic document 
application is received. An annotation template for the requested annotation is retrieved. The 
5 annotation template is external to the electronic document application and the electronic 

document and includes fixed content, associated instructions defining variable content, and 
layout information for the fixed content and variable content. The associated instructions are 
executed to define the variable content for an instance of the requested annotation. The instance 
of the annotation is applied to the electronic document, such that the annotation instance has an 

10 appearance defined by the variable content, the fixed content and the layout information. 

Particular implementations can include one or more of the following features. A 
condition based on a state of a system, the electronic document application or the electronic 
document can be determined and the variable content can be based on the determined condition. 
The instructions to define variable content can generate graphics or obtain graphics, text, an 

15 image, audio, video or a hyperlink. The appearance of vector art can be defined. The 

instructions can define a layout for the variable and fixed content. Content information can be 
received from a remote or local system or memory. The annotation template can include an 
electronic form and the electronic form can have form fields with associated scripts. The 
annotation content can be flattened to create a static instance of the annotation and the instance 

20 can be applied to the electronic document to create a fixed representation of the instance of the 
annotation. When the instance of the annotation is applied, the instance can be associated with a 
specific location in the electronic document. 

The invention can be implemented to provide one or more of the following advantages. 
An annotation template can be created that includes fixed and variable content, and layout 

25 information that defines the spatial relationship between content. The fixed content includes 
content that will be incorporated in each instance of an annotation. The variable content is 
content that is dynamically generated at the time the annotation is applied, and can include, for 
example, dynamically generated text or graphics. The layout of the annotation content can be 
dynamically determined. The annotation content can be automatically retrieved from a source 

30 outside the electronic document or document application. For example, the logic that performs 
the steps to create the annotation can include instructions that cause the system to find and 
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integrate data from a server into the annotation. Annotation templates can be added, edited and 
deleted by an author without altering the application, making the annotations fully customizable. 
The instructions to create the annotation are not a part of the electronic document application's 
logic that adds the annotation to the electronic document. The annotation template is also not a 
part of the document to which the annotation is being applied. The annotation templates can be 
customized for the end user, making each annotation template useful in a particular instance. 
The appearance of the annotation can depend on factors or values related to the document to 
which it is applied or the appearance can simultaneously or separately depend on factors 
independent of the electronic document. Because the annotation is instantiated just before 
placement, different instances of an annotation of the same type can have different appearances 
depending on conditions when the annotation is applied. 

The details of one or more embodiments of the invention are set forth in the 
accompanying drawings and the description below. Other features, objects, and advantages of 
the invention will be apparent from the description and drawings, and from the claims. 

DESCRIPTION OF DRAWINGS 

FIG. 1 is a flow chart illustrating a method of adding an annotation to a document. 
FIG. 2 illustrates an exemplary annotation. 

FIG. 3 shows an example of a document to which an annotation can be applied. 

FIG. 4 illustrates an annotation template with instructions to create the annotation content. 

FIG. 5 shows the document with a flattened instance of the annotation. 

Like reference symbols in the various drawings indicate like elements. 

DETAILED DESCRIPTION 

The invention provides techniques for defining annotations in the context of electronic 
documents. An annotation is an object that can be added to an electronic document to comment 
on, explain, expand upon, or otherwise mark the content of the electronic document, often in a 
way that mimics annotations in conventional paper documents. Thus, for example, an annotation 
can embody a reader's commentary on a particular passage of an electronic book, resembling 
notes penciled in the margin of or a "sticky note" attached to a page of a conventional paper 

book. Similarly, an annotation can provide a stamp that, when applied at a particular location in 
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an electronic document, identifies content at that location as "Draft" or "Confidential", just as 
might be done with a conventional rubber stamp in a paper document. Like conventional 
annotations, electronic annotations can include text and graphics; unlike their conventional 
counterparts, however, electronic annotations can include any other form of electronic content, 
5 including, for example, audio and video information such as sounds and movies. 

The annotations described herein have an appearance that is determined by defining some 
of the annotation content at the time the annotation is applied. To create this content, the 
annotation includes a series of instructions, for example, a script that can be executed to create an 
annotation instance based on data available when the annotation is selected to be applied. 

10 FIG. 1 illustrates a method 100 for adding an annotation to an electronic document that 

can be implemented in an electronic document application according to one aspect of the 
invention. The method begins with a request to apply an annotation to an electronic document 
(step 110). A user can request the annotation by selecting a particular annotation type - for 
example, by selecting a tab, button, option from a pull down menu or other user interface that 

15 identifies annotation types that are available for the particular document or application. 

Alternatively, the request can be generated automatically in response to an event, such as the 
opening of a document. The request can specify a particular annotation type or a generic 
annotation of unspecified type. If the request is generic, the system can prompt the user to 
identify the type of annotation to be applied to the document - for example, by presenting a pull 

20 down menu, a list with check boxes or radio buttons, or some other interactive user interface 
through which the user can select a desired annotation type. 

Each annotation type can have an associated annotation template that is used to define the 
appearance of instances of the corresponding type, as will be described in more detail below. An 
annotation template is an object, such as a file, that includes the instructions and optionally other 

25 information, including fixed content, such as vector graphics, and layout information, that will be 
used to define the appearance of an instance of the annotation. The annotation templates are 
external to the application and the document - i.e., the templates are not coded directly in the 
application and do not reside in the document to which the annotations are being applied. The 
templates can therefore be stored and manipulated separate from the application and document, 

30 as will be discussed in more detail below. The application can determine what annotation types 
are available by querying annotation template storage and presenting the available types for 
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selection by the user. For example, the application can query a particular directory or a database 
in which annotation templates are stored. 

In response to the request, the application retrieves fixed content, layout information and 
annotation instructions that are associated with the annotation type (step 120). The instructions 
5 define some of the content of the annotation, as described in more detail below. Additionally, 
the instructions can define in whole or in part how the content is to be arranged or implemented. 
For example, the instructions can determine some or all of the layout and the appearance of the 
content (e.g., instead of or in addition to any layout or appearance information defined as a fixed 
part of the annotation template). The application can retrieve the instructions in the form of a 
10 script, such as a script in the JavaScript language or another machine readable code. The 
instructions for each annotation type can be stored (and retrieved) in a unique annotation 
template. 

In one implementation, the annotation template can be an electronic form. Forms 
typically include fields that can have associated instructions that specify actions to be performed 

15 in the form, such as calculating a value or values to be entered in a form field, performing an 

action (e.g., retrieving data from a specified location) in response to user input (e.g., selection of 
a button by a user), or the like. Forms can also be used to transmit data and receive a response. 
Thus, annotation templates can be implemented as files in a format that supports scriptable 
electronic forms, such as the Portable Document Format ("PDF"), introduced by Adobe Systems 

20 Incorporated of San Jose, California. Annotation templates can also include static content that 
will determine a portion of the appearance (e.g., a background) of annotations instantiated from 
the type. 

The application interprets and/or executes the instructions to create content for a specific 
instance of the annotation (step 130). Executing the instructions can cause the application to 

25 define the variable content, generally by performing any combination of creating, retrieving, 

arranging, or transforming of data or elements from a source where some or all of the elements 
are stored, such as the system's memory, external memory, a server or other remote location. 
Because the annotation is instantiated with content and in some cases a layout, as described 
below, that is defined at the time the annotation is applied, each instance of a particular 

30 annotation type can be created with a different appearance, determined by a condition or 
conditions at the time it is created, as further described below. 
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The application creates an instance of the annotation according to the fixed layout 
information included in the annotation template and, if available, variable layout information 
defined by the instructions (step 140). The layout information defines the relationships between 
the different pieces of fixed, variable or fixed and variable content. The fixed content and the 
5 variable content are arranged according to the layout information. In one implementation, where 
the annotation is implemented as an electronic form, creating an annotation instance can include 
defining an instance of the form having defined values for one or more form fields provided in a 
template form. 

The annotation instance is applied to the document (step 150). To apply the annotation 

10 instance, the defined instance is flattened, and the flattened instance is applied to the document. 
Flattening can include, for example, converting the now-defined annotation content, as well as 
any fixed content, into a fully static form, such as a bitmap. Generally, the flattened annotation 
cannot be transformed back into variable content. In some implementations, the user can be 
provided the option of changing the annotation before the annotation is flattened. Once the 

15 annotation is flattened, the annotation cannot be updated when an element on which the 

annotation was based changes, unlike form fields which may change when a change occurs in the 
document. Application of the annotation instance can include associating the annotation instance 
with the electronic document and storing the annotation instance in association with (e.g., within) 
the document. For example, if the document is a PDF document, the annotation instance can be 

20 added as a layer to the PDF document. 

Thereafter, when the document is rendered, a representation of the annotation instance is 
rendered at the location at which the annotation was applied. The application can place the 
annotation instance in a variety of ways - for example, in response to user input specifying a 
location for the annotation, based on information included in the original request to apply the 

25 annotation, according to a value or rule (e.g., defined by the instructions), or in a predetermined 
location. For example, if the annotation is placed based on the request, a reference point (e.g., 
the annotation's center) can be located at a location on the document page where the user 
activates the pointing device. Alternatively, the application can invoke a rule that defines a 
location for the annotation - for example, a rule defining an offset distance from the location at 

30 which the pointing device was activated. In still another alternative, the instructions themselves 
can define a location for the annotation based on external conditions, such as the number of 
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times an annotation of the particular annotation type has been previously applied to the electronic 
document. In some implementations, once the annotation is applied the user can modify the 
annotation - for example, by moving the annotation to another location or deleting the 
annotation. In particular implementations, the ability to modify annotations can be restricted 

5 according to rights or permissions, such as permissions provided with the application or with the 
particular document. 

In some implementations, a request need not be received from a user, but can be 
generated automatically, such as in response to an event. Consider, for example, a document that 
is to be reviewed by more than one user before it is completed. The document can be created by 

10 a first user who includes, as a part of the document, a request to apply an annotation each time 
the document is opened. Generally the request specifies a specific annotation template — for 
example, a template for an annotation that includes an identification of the user opening the 
document and the date. Thus, each time that a user opens the document, the annotation template 
is retrieved and the instructions are executed to define variable content including the date and 

15 user identification. An annotation instance is created, as described above, and applied to the 
document. Each of these annotations is displayed in the document - for example, on the first 
page - such that each subsequent user who views the document can readily identify all other 
users who have previously viewed the document, and when the document was viewed. 

The instructions for creating the annotation content can cause the application to retrieve 

20 data from a variety of locations, such as the system's memory, external memory, a server or 

other remote location over a network, the document that the annotation is to be applied to, or the 
user can be prompted to enter information. The data can include any information available to the 
application, such as the identity of the application user, the date, the document type, the 
document name, the number or contents of the page on which the annotation is going to be 

25 applied, or a serial number or other information retrieved from a local or remote location. The 
application can use the retrieved data to dynamically define annotation content based on one or 
more conditions or rules. For example, the application can define the variable content based on a 
condition related to the state of the document, such as how many times the document has been 
opened, how many annotations have been applied or how many other users have annotated the 

30 document. Alternatively, the condition can be unrelated to the document, or external to the 
document, such as the date, the time, or a number generated by the application or another 
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application. Some of the content can be defined according to the condition(s) or retrieved 
information, while some portion of the content can be predefined (i.e., fixed). 

The fixed or variable annotation content can include any item that can be presented in an 
electronic document, such as graphics, text, images, audio, video, hyperlinks, or a combination 
of two or more of these. Audio and video content can either be in an open (activated) or closed 
state when the document is opened. If the content is not activated automatically upon opening 
the document, a user can activate the audio or video by selecting the annotation. Hyperlinks 
generally must be activated by the user to jump to a linked object. The graphics can be in any 
suitable form for the application, such as bitmaps, vector art, or raster art. If the appearance of 
no fixed content is desired, the fixed content can include a transparent frame. 

As discussed above, the template can include layout information that defines the spatial 
relationships between the content. Because this layout information is included in the annotation 
template, each time an annotation is rendered from a particular annotation template, the layout 
information causes the application to configure this content in the same way each time. The 
application template can additionally include instructions defining the annotation layout based on 
the conditions, such that how the content will be presented is defined and can change from one 
instance of applying the annotation to another. As an example, an annotation template can 
include instructions specifying a variable layout for an annotation including a text object and a 
graphic object, where the relationship of the two objects changes with each application of the 
annotation moving the text in a clockwise direction in relationship to the graphic each time the 
annotation is applied. In this example, when the annotation is first applied in a document, the 
text is positioned in one orientation relative to the graphic - such as above the graphic. When a 
second annotation is applied in the document using the same annotation template, the text is 
positioned in a different orientation relative to the graphic - e.g., to the right of the graphic. A 
global variable can be used to track the number of times the annotation is applied. Other aspects 
of the appearance of the content, such as the color, size, shape and effects applied to the content, 
can also be determined by the conditions. 

In some implementations, the application can be configured to enable authors to define 
new dynamic annotation types. The author defines a new annotation type by defining an 
annotation template. In one implementation, where the annotation template is an electronic 
form, the author creates the annotation template using conventional form creation techniques, 
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defining one or more form fields with associated instructions, such as scripts, that will be used to 
define content for the associated fields when an annotation of the type is to be applied. Because 
the annotation instructions are external to the application, they can be modified by simply 
modifying the annotation template or file using conventional applications, such as word 
5 processors or Adobe® Acrobat® software, without requiring modifications to the application 
code. 

One specific example of applying an annotation to an electronic document will now be 
described, with reference to FIGS. 2-5. In the following example, an annotation 200 including a 
serial number 205, a user identifier 210 and a date 215, can be applied to an electronic document, 

10 as shown in FIG. 2. The serial number 205 is a unique number that is received from a 

centralized database, such that multiple users can access the database and each user can apply a 
unique number. Because the information compiled to create the annotation is retrieved just 
before application, the annotation will include the most current information available, which is 
the next number in the series. 

15 As shown in FIG. 3, the user begins by opening the electronic document 300 in an 

electronic document application. To select an annotation to be applied in the document, the user 
selects a desired annotation tool - for example, by selecting a corresponding menu item from a 
pull down menu that identifies all annotation types that are supported by the application and/or 
the document 300. In this example, the user selects the annotation type "serial number". When 

20 the user has selected the desired annotation type, the application changes the appearance of the 
cursor to indicate that an annotation tool has been selected. The user then specifies where the 
annotation will be placed by navigating the cursor 305 over a desired paragraph and activating 
the pointer tool. 

The application receives the selection and retrieves an annotation template 400 associated 
25 with the "serial number" annotation, as shown in FIG. 4. The template 400 contains the fixed 
content, a set of instructions that can be executed to define the variable content, and layout 
information. The application executes a first instruction 405 to retrieve a serial number from a 
database. A second instruction 410 is executed, which determines a user identifier. In this 
example, each user identifier includes a job title and number and is a part of the user's login 
30 information. The third instruction 415 causes the current time at which the annotation is 
instantiated to be retrieved. 
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In addition to the instructions that determine the variable content, the template also 
includes fixed content 420. In this case, the fixed content 420 defines a rectangular yellow 
frame. Referring back to FIG. 2, the layout information 425 specifies that the serial number 205 
should be placed at the upper center, the user identifier 210 in the middle and the date 215 on the 
bottom center of the text field. The variable content is then centered on the fixed content 220. In 
this example, the instructions do not specify additional layout information. 

Once all of the instructions that determine the appearance of the annotation have been 
performed, the annotation content is flattened to define a static appearance of the annotation. 
The application performs a script that designates the placement of the annotation, which in this 
example applies the annotation an inch to the right of the paragraph in which the cursor was 
located when the annotation was selected. The flattened annotation is associated with this 
location so that each time the document is rendered the annotation appears at the same location. 
The document, including a layer corresponding to the serial number annotation, can be saved for 
future access. 

As shown in FIG. 5, when the user or another viewer later opens the annotated electronic 
document 500, the annotation is displayed at the location at which it was previously applied. 

The invention can be implemented in digital electronic circuitry, or in computer 
hardware, firmware, software, or in combinations of them. The invention can be implemented as 
a computer program product, i.e., a computer program tangibly embodied in an information 
carrier, e.g., in a machine-readable storage device or in a propagated signal, for execution by, or 
to control the operation of, data processing apparatus, e.g., a programmable processor, a 
computer, or multiple computers. A computer program can be written in any form of 
programming language, including compiled or interpreted languages, and it can be deployed in 
any form, including as a stand-alone program or as a module, component, subroutine, or other 
unit suitable for use in a computing environment. A computer program can be deployed to be 
executed on one computer or on multiple computers at one site or distributed across multiple 
sites and interconnected by a communication network. 

Method steps of the invention can be performed by one or more programmable 
processors executing a computer program to perform functions of the invention by operating on 
input data and generating output. Method steps can also be performed by, and apparatus of the 
invention can be implemented as, special purpose logic circuitry, e.g., an FPGA (field 
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programmable gate array) or an ASIC (application-specific integrated circuit). Processors 
suitable for the execution of a computer program include, by way of example, both general and 
special purpose microprocessors, and any one or more processors of any kind of digital 
computer. Generally, a processor will receive instructions and data from a read-only memory or 
a random access memory or both. The essential elements of a computer are a processor for 
executing instructions and one or more memory devices for storing instructions and data. 
Generally, a computer will also include, or be operatively coupled to receive data from or 
transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, 
magneto-optical disks, or optical disks. Information carriers suitable for embodying computer 
program instructions and data include all forms of non- volatile memory, including by way of 
example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; 
magnetic disks, e.g., internal hard disks or removable disks; magneto-optical disks; and CD- 
ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or 
incorporated in special purpose logic circuitry. 

A number of embodiments of the invention have been described. Nevertheless, it will be 
understood that various modifications may be made without departing from the spirit and scope 
of the invention. For example, other commands can be executed by the annotation type's script, 
including setting button form fields to an arbitrary appearance, changing text, changing form 
fields, changing the appearance of vector art-based form fields and moving form fields. The 
shape of the annotation can be non-rectangular. The annotation can be scalable. The annotation 
can be chosen by selecting a button that causes the actions to be performed. The actions can call 
the commands that create an annotation. Accordingly, other implementations are within the 
scope of the following claims. 
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